32 research outputs found
Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations
Model-agnostic interpretation techniques allow us to explain the behavior of
any predictive model. Due to different notations and terminology, it is
difficult to see how they are related. A unified view on these methods has been
missing. We present the generalized SIPA (sampling, intervention, prediction,
aggregation) framework of work stages for model-agnostic interpretations and
demonstrate how several prominent methods for feature effects can be embedded
into the proposed framework. Furthermore, we extend the framework to feature
importance computations by pointing out how variance-based and
performance-based importance measures are based on the same work stages. The
SIPA framework reduces the diverse set of model-agnostic techniques to a single
methodology and establishes a common terminology to discuss them in future
work
Visualizing the Feature Importance for Black Box Models
In recent years, a large amount of model-agnostic methods to improve the
transparency, trustability and interpretability of machine learning models have
been developed. We introduce local feature importance as a local version of a
recent model-agnostic global feature importance method. Based on local feature
importance, we propose two visual tools: partial importance (PI) and individual
conditional importance (ICI) plots which visualize how changes in a feature
affect the model performance on average, as well as for individual
observations. Our proposed methods are related to partial dependence (PD) and
individual conditional expectation (ICE) plots, but visualize the expected
(conditional) feature importance instead of the expected (conditional)
prediction. Furthermore, we show that averaging ICI curves across observations
yields a PI curve, and integrating the PI curve with respect to the
distribution of the considered feature results in the global feature
importance. Another contribution of our paper is the Shapley feature
importance, which fairly distributes the overall performance of a model among
the features according to the marginal contributions and which can be used to
compare the feature importance across different models.Comment: To Appear in Machine Learning and Knowledge Discovery in Databases:
European Conference, ECML PKDD 2018, Dublin, Ireland, September 10 to 14,
2018, Proceedings, Part
Inverse Classification for Comparison-based Interpretability in Machine Learning
In the context of post-hoc interpretability, this paper addresses the task of
explaining the prediction of a classifier, considering the case where no
information is available, neither on the classifier itself, nor on the
processed data (neither the training nor the test data). It proposes an
instance-based approach whose principle consists in determining the minimal
changes needed to alter a prediction: given a data point whose classification
must be explained, the proposed method consists in identifying a close
neighbour classified differently, where the closeness definition integrates a
sparsity constraint. This principle is implemented using observation generation
in the Growing Spheres algorithm. Experimental results on two datasets
illustrate the relevance of the proposed approach that can be used to gain
knowledge about the classifier.Comment: preprin
Explanations of Black-Box Model Predictions by Contextual Importance and Utility
The significant advances in autonomous systems together with an immensely
wider application domain have increased the need for trustable intelligent
systems. Explainable artificial intelligence is gaining considerable attention
among researchers and developers to address this requirement. Although there is
an increasing number of works on interpretable and transparent machine learning
algorithms, they are mostly intended for the technical users. Explanations for
the end-user have been neglected in many usable and practical applications. In
this work, we present the Contextual Importance (CI) and Contextual Utility
(CU) concepts to extract explanations that are easily understandable by experts
as well as novice users. This method explains the prediction results without
transforming the model into an interpretable one. We present an example of
providing explanations for linear and non-linear models to demonstrate the
generalizability of the method. CI and CU are numerical values that can be
represented to the user in visuals and natural language form to justify actions
and explain reasoning for individual instances, situations, and contexts. We
show the utility of explanations in car selection example and Iris flower
classification by presenting complete (i.e. the causes of an individual
prediction) and contrastive explanation (i.e. contrasting instance against the
instance of interest). The experimental results show the feasibility and
validity of the provided explanation methods
Local Interpretation Methods to Machine Learning Using the Domain of the Feature Space
As machine learning becomes an important part of many real world applications
affecting human lives, new requirements, besides high predictive accuracy,
become important. One important requirement is transparency, which has been
associated with model interpretability. Many machine learning algorithms induce
models difficult to interpret, named black box. Moreover, people have
difficulty to trust models that cannot be explained. In particular for machine
learning, many groups are investigating new methods able to explain black box
models. These methods usually look inside the black models to explain their
inner work. By doing so, they allow the interpretation of the decision making
process used by black box models. Among the recently proposed model
interpretation methods, there is a group, named local estimators, which are
designed to explain how the label of particular instance is predicted. For
such, they induce interpretable models on the neighborhood of the instance to
be explained. Local estimators have been successfully used to explain specific
predictions. Although they provide some degree of model interpretability, it is
still not clear what is the best way to implement and apply them. Open
questions include: how to best define the neighborhood of an instance? How to
control the trade-off between the accuracy of the interpretation method and its
interpretability? How to make the obtained solution robust to small variations
on the instance to be explained? To answer to these questions, we propose and
investigate two strategies: (i) using data instance properties to provide
improved explanations, and (ii) making sure that the neighborhood of an
instance is properly defined by taking the geometry of the domain of the
feature space into account. We evaluate these strategies in a regression task
and present experimental results that show that they can improve local
explanations
ICIE 1.0:a novel tool for interactive contextual interaction explanations
With the rise of new laws around privacy and awareness, explanation of automated decision making becomes increasingly important. Nowadays, machine learning models are used to aid experts in domains such as banking and insurance to find suspicious transactions, approve loans and credit card applications. Companies using such systems have to be able to provide the rationale behind their decisions; blindly relying on the trained model is not sufficient. There are currently a number of methods that provide insights in models and their decisions, but often they are either good at showing global or local behavior. Global behavior is often too complex to visualize or comprehend, so approximations are shown, and visualizing local behavior is often misleading as it is difficult to define what local exactly means (i.e. our methods don’t “know” how easily a feature-value can be changed; which ones are flexible, and which ones are static). We introduce the ICIE framework (Interactive Contextual Interaction Explanations) which enables users to view explanations of individual instances under different contexts. We will see that various contexts for the same case lead to different explanations, revealing different feature interactions.</p
Meaningful Data Sampling for a Faithful Local Explanation Method
Data sampling has an important role in the majority of local explanation methods. Generating neighborhood samples using either the Gaussian distribution or the distribution of training data is a widely-used procedure in the tabular data case. Generally, this approach has several weaknesses: first, it produces a uniform data which may not represent the actual distribution of samples; second, disregarding the interaction between features tends to create unlikely samples; and third, it may fail to define a compact and diverse locality for the sample being explained. In this paper, we propose a sampling methodology based on observation-level feature importance to derive more meaningful perturbed samples. To evaluate the efficiency of the proposed approach we applied it to the LIME explanation method. The conducted experiments demonstrate considerable improvements in terms of fidelity and explainability